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ABSTRACT 

Content analysis is a systematic and objective 
technique which reduces into smaller sub-parts existing 
communications* It is the analytical reduction of a text to a 
standard set of statistically manipulatable symbols representing the 
presence, intensity, or frequency of characteristics. A case study of 
the use of content analysis is the U.S. Forest Service's formulation 
in the early 1970s of a development policy for the Shawnee National 
Forest in southern Illinois. The alternatives were: no projects, new 
limited access highway, combination scenic drive and trail, national 
scenic trail, and improve existing roads. Three hundred documents 
expressing public opinion were analyzed according to various 
categories, given a numerical score, and cross tabulated. On the 
basis of the assumption that people's attitudes regarding the 
alternative were related to underlying concerns and factors, which 
when subjected to multiple correlation analysis could be used to 
predict attitudes, the content analysis verified the decision to opt 
for the scenic route over the existing road system. As a method of 
describing, understanding and explaining, predicting, and deciding, 
especially in cases where available data is not in the form required, 
content analysis is a reliable and valid research method and central 
to the development of adult education. (Author/JR) 
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r-l 

r-i Many data now exist which to varying degrees are relatively 

y I useless for evaluation and research purposes. These data exist as 

logs, diaries, newspaper articles, resolutions, and other narrative 
communications. They are organized for their original purpose but 
unorganized for research purposes. They are adequate in quantity for 
original purposes but become too voluminous when combined. They, 
therefore, need reductions to manageable forms. Content analysis 
is a method which can serve these functions. 

What is Content Analysis ? 

Content is central in the communication and education process. 
Communication content is the body of meanings conveyed through symbols, 
which make up the communications itself. 

Since communication content is so influenced by the countless 
aspects of human experience, and its causes so varied, a systematic 
method has been developed — and is being further developed — to describe 
the various facets of communication content in summary fashion. 
This method is called content analysis. 

A review of several definitions in the literature identified six 
major characteristics of content analysis: 

1. It applies only to social science generalizations. (Leites and Pool 1942) 



2. It applies only or primarily to determining the effects of 

communication (Waples and Berelson 1941) 

3. It applies only to the syntactic and semantic dimensions of language. 

(Lietes and Pool 1942) 
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4. It must be Objective. (Waples and Berelson 1941, Leites and 

Pool 1942, Janis 1943, Kaplan 1943.) 

5. It must be Systematic. (Leites and Pool 1942, Kapland and 

Goldsen 1949, Kaplan 1943) 

6. It must be Quantitative. (Waples and Berelson 1941, Leites 

and Pool 1942, Kapland and Goldsen 
1949, Janis 1943, Kaplan 1943) 

Anderson etal (1975) say content analysis is a general assessment 

technique by which complex phenomena (e.g compositions, adult 

conversations, text books.) can be reduced to simpler terms, (e.g. 
work frequency counts, categorizations of the content of conversations, 
readability scores.) p. 82. It can be modified to obtain many types 
of research and evaluation data. 

To summarize, content analysis is a technique for systematic, objective 
and quantitative description of communications for use in the social 
sciences including adult education. It provides reduction of existing, 
written communications data such as letters, and resolutions to simpler 
and fewer terms. These fewer terms are primarily numerical counts of 
the existence of certain expressions, but also the frequency and 
intensity to which those expressions are present. 

We've found the technique to have demonstrable use in analyzing 
a situation, needing decision making on planning alternatives, i.e., 
a situation which had inherent potential for adult education program 
development. We have found several purposes can be served by the 
technique. This paper summarizes these experiences in that situation 
illustrates those purposes, and discusses further general applications. 

A Case Study Using Content Analysis 

In the early 1970's, the U.S. Forest Service faced' the problem of 

ERIC 003 



-3- 



deciding on the future development policy in the Shawnee National 
For;*st, in southern Illinois • Specifically its question was: should 
future policy aim at: 1) preserving the Shawnee as it is, 2) enhancing 
wider usage of the Shawnee's resources, 3) providing economic benefits 
to southern Illinois and the surrounding area, 4) helping people enjoy 
the aesthetic and ecological offerings of the Shawnee or 5) a combination 
of these alternatives. (Forest and Flitter, 1974) 

Within these policy questions, the Forest Service, in 1973 needed 
to decide between specific, immediate transportation and usage alternatives* 

1. Develop no projects 

2. New limited access highway 

3. Combination scenic drive and trail 

4. National scenic trail 

5. Improve existing roads, and 

6. Other alternatives 

As described by the PDK study committee on evaluation (1971) , 
The Forest Services decision-making occurred in four stages: 1) becoming 
aware of a needed decision; 2) designing the decision situation; 
3) choosing among alternatives; and 4) acting upon the chosen alternative. 

Much of the first stage was completed by the time this study began. 
This study dealt with the second and third stages of the decision making 
process. Stage four is reported herein. 

The specific purpose of this project thus became: to determine 
the attitudes of interested publics toward the six planning alternatives, 
and the factors rel'ated to these attitudes, so that the U.S. Forest 
Service could give some consideration to public input in deciding on 
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which alternatives to implement. The methods of data collection, 
analysis, interpretation, and conclusions are summarized below. 

Summary of Study Procedures 

1. Public hearings were held (June and September, 1973) in southern 
and central Illinois. These hearings were transcribed. The 
transcriptions and prepared statements became data for analysis. 

2. Forest Service files at the Shawnee Supervisor's office were 
searched. Related letters and other documents from the later 
1960 's and early 1970 's were included in study. 

3. Forest Service invited opinions and expressions of support or 
disagreement, and reasons from key persons they worked with, 
(i.e., community leaders. Sierra Club, conservancy districts, etc.) 

4. Newspapers were searched for relevant content, (feature articles, 
editorials, news articles from Chicago, and local southern Illinois 
papers) 

5. Copies of state and federal legislative actions were secured, (bills, 
resolutions) 

6. The 300 eventual documents were numbered, duplicated and divided 
into eight categories plus miscellaneous. 



Category Number 

1. Editorials and letters to editors 3 

2. Feature articles and news articles 84 

3. News releases from Forest Service 4 

4. Transcripts of public meetings 50 

5. Legislation correspondence 21 
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Category Number 

6. Unsolicited letters from citizens 101 

7. Bills and resolutions 19 

8. Petitions 2 

9. Miscellaneous 18 



7. A coding form and a key sheet were developed. They were categorized 
to be compatible with computer card orientation. Answers on the 
code sheet were directly key punched for computer programming. 
(See appendix A for specific variables) The general variables coded 
by the content analyzers were: 

a. Identification of the document 

b. Document type 

c. Content analyzer's identification 

d. Date of the documents 

e. Expression of approval or disapproval of each of the six possible 
alternatives • 

f. Factors exp*-*»ssed by choice of respondent as reasons or 
concerns in their alternative. 

g. Population affiliations, sex, and occupation of the population 
respondents. 

8. Seven judges" or content analyzers were chosen from varied backgrounds 
who did not previously know each other and had independent, different 
professional training. (educatory, economics, soils, recreation, and 
environment) This panel of analyzers objectified their perceptions, 
reduced bias and their scores provided a chance to check reliability 
of their perceptions. 
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9. The seven content analyzers were 
in group sessions by the project 
were consistent. All were given 



trained to do content analysis 
directors, so that all instructions 
the same interpretations, directions, 



and explanations of the project, its objectives, their tasks, the 
code sheets and the key sheets. 
10. The seven content analyzers worked independently in coding the 
300 document., according to the key sheet. 
The procedure used by each content analyzer was as follows: 
They would take a letter such as: 
Dear Mr. Hendricks: 

As a life-long resident of southern Illinois, 
I believe it is about time, the Shawnee Forest served 
the needs of the local people. We need economic and 
business development here, but we also need our 
beautiful natural resources preserved. If we don't 
have those we don't have anything. 

Therefore, I feel you should not build a new 
super highway through our area. If tourists need 
a place to drive, help us improve our existing roads. 
I strongly urge you to do this in considering our needs. 

Sincerely, 



Marcus Anderson 

Each analyzer would look at this document and decide if the 
respondent was advocating a position. In the above example Marcus 
Anderson was obviously advocating the improvement of existing road alternative. 
The analyzer would judge how strongly the person was for it. In Mr. 
Anderson's case, a judge might interpret his feelings strongly positive 
and circle 5 for alternative V. The other six judges might have 
given 4, 5,3, 2, 5, and 5 to the letter regarding alternative V. 
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All 45 variables on the score sheet were scored this way by 
each content analyzer. If expression or data on certain variables were 
not perceived to be in the document, the content analyzer would circle 
(0) zero on the score sheet for that variable. 

11. Once each of the content analyzers had completed their analysis of 
the 300 documents, statistical analyses were undertaken. First, 
the judgments of each analyzer on each variable for each document 
were averaged so that each document would have but one score on 
each of the 45 variables. 

12. When the reactions of the seven- judge panel toward each of the 

45 variables were averaged into one scorei the final stage of analysis 
used the 45 scores for each of the 300 documents. These scores 
gave us the attitude summarise* and the respondent characteristics 
related to their attitudes. 

13. Statistical procedures used to summarize, describe, analyze, and 
interpret the data included: 

1. Frequency and percentar^.e distributions in each of the nine 
categories Certain categories were limited to this treatment 
because of limited numbers of documents in those categories* 
30 to 40 documents in a category were the minimum for further 
analysis. 

2. Cross tabulations, Pearson's Product-Moment-Correlation» (r) , 
and Multiple Correlation (R) were computed on certain data 
to see which factors and/or clusters of factors were related 
to attitudes towards the alternatives. An arbitrary minimum 
of .400 was set as an acceptable Multiple R. 
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14. Computerized results were analyzed and compiled in report form, 
as a rough draft for reaction by other persons. 

15. The final draft of the study was presented to the U.S. Forest Service. 

RESULTS 

Reliability of the content analyzers was determined by determining 
the degree to which each judge agreed with the majority of judges 
on each question. The percentages of how much each analyzer was 
congruent with the other six were as in Table 1. 



Table 1 

Reliability of Content Analyzers 



Analyzer 


% Consruence 


1 


75.3 


2 


86.2 


3 


78.0 


4 


84.2 


5 


79.9 


6 


77.3 


7 


81.5 
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towards 



The content of these 300 documents when analyzed showed attitudes 
lis the alternatives as shown in Table 9 i-ii-uaes 



shown in Table 2. 
TABLE 2 



Summary of Attitude Frequencies Towards the Alternatives 



PlanniiiK Alternative 


Attitudes ^ 


Negative Nmifral d^c4^4„_ 


1. Do Nothing 


37 


236 


27 


2. Limited access highway 


40 


50 


210 


3. Combination Road 
and trail 


A 


284 


12 1 


A* Scenic trail 

> 


3 


286 


i 

11 ; 

t 

} 


5. Improving existing roads 

As shown, the only alternative 


5 

receiving subs 


251 

tantial expre 


44 

ssions 



of positive attitudes was the limited access highway (alternative 2) . 
Expressions toward the other alternatives were generally neutral or 
nonexistent. 

As stated, data on other variables besides attitudes were determined 
through analysis, "nderstanding these was very important to the complete 
pictiiro nnd nipauLng of cho iittitvides. 
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1. The analysis iletormlned people in 1973 were much more positive 
toward the alternatives of "doing nothing" and "improving existing 
roads", and more vocally negative towards the limited access highway, 
than they were in previous years. 

2. Attitudes depend on or relate to certain underlying concerns 
and factors. The factors of land, water, and nature based recreation; 
community needs; individual needs; the economy; politics; mining; logging; 
travel and transportation; education and science, and protection of 

the environment all influenced the thinking and attitudes of people. 
The specific effects can only be cited by referring to the specific 
alternatives and data category. These concerns must be part of the 
summary, analysis, and usage of the attitudes. Some of these 
relationships as existed in the public hearings transcript content 
are presented in Table 3. 

As Table 3 shows, certain concerns were negatively related (i.e., 
expressions in documents on concern for the protection of the environment, 
were negatively correlated -.684 with positive attitudes towards limited 
access highway). Expressions of concern for the economy, however, 
showed a positive correlation of .555 with positive attitudes towards 
the new highway alternative. 
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TABLE 3 

Factors Related to Positive Attitudes 



Expressed 

Positive attitudes toward: 



Relates to 



1. Do nothing 



Expressed Concern For "r" 

Community needs -.419 
Individual needs -•353 
Economy -.513 
Education .499 
Protection of environment .461 



2. Limited access highway Water recreation .375 

Community needs .508 
Economy ' .555 

Education -.401 
Protection of environment -^.684 



3. Combination road and trail Water recreation .381 

Conmiunity needs . .410 
Economy .290 



4. Scenic trail No factors related 
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Community Needs .293 
Transportation .284 
Protection of environment .376 
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When all expressed concerns or factors determined through analysis 
were combined to see which clusters of concerns most efficiently predicted 
attitudes towards the alternatives, the clusters in Table 4 with accordant 
multiple correlations resulted. 

TABLE 4 

Multiple Correlations of Alternatives and Concerns 

" Z_ ■ : ■ . 2^' 

Alternative ^ S^cpressed Concerns Multiple "R" 

1. Do Nothing Economy .789 

Individual needs 
Education 
Politics 

Protection of environment 



2. Limited access highway Protection of environment •BIB 

Coiuiunity needs 
Economy 



3. CombinaLion road and trail Water recreation .584 

Community needs 
Education 

Protection of environment 
Individual needs 
Transportation 



No Strong 

4. Scenic trail None Multiple '^R" 



5. Improve existing roads Community heeds .617 

Individual needs 
Protection of environment 
Education 
Politics 
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As Table 4 shows, a multiple correlation of .789 resulted between 
five clustered concerns (the economy, individual needs, education, 
politics, and protection of environment) and the do nothing alternative. 
In other words, to know a person was concerned for all five of these 
concerns allows us to predict his/her attitude toward the do nothing 
alternative with relative certainty. With even more certainty, expressed 
concerns (or lack of them) for the environment, the economy, and community 
needs predict attitudes toward the limited access highway with a .818 
Multiple "R". 

The transcripts of public hearings more than any other category 
identified factors which could be considered underlying reasons or 
concerns related to peoples' attitudes. Knowing the importance, those 
participating in the hearings attached to such factors as the economy, 
protection of environment., community needs, transportation, and water 
recreation allowed us to know with a fair certainty how they viewed the 
five alternatives and why. 

Summary of Study Results 

1. The limited access highway alternative without question had more 
expressed positive attitudes than any other alternative. 

2. The limited access highway was however, the most controversial or 
polarized of the alternatives. Though many people expressed positive 
attitudes toward it, many others expressed negative attitudes and 
people were unlikely to remain neutral on this alternative. 

3. Attitudes toward the other four alternatives were primarily neutral, 
most likely reflecting disinterest or lack of awareness. 

4. The attitudes towards the five alternatives were very dynamically 
changing at the time of the study. Specifically, the positive 
attitudes towards the limited access highway are declining 
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while the negative attitudes are increasing. People areaalsc 
becoming less neutral and more positive towards the alternatives 
of do nothing and improvement o£ existing roads. It may be several 
years before all persons of the types whose expressed attitudes 
were analyzed here have crystallized an attitude or position 
towards each of the alternatives. 

5. If the attitudes of the public were to be considered in the decision, 
their underlying concerns or reasons for their attitudes must be 
considered. They were considered by the U.S. Forest Service. 

6. Forest supervisor, Charles Hendricks, announced on August 23, 197A 
that the Forest Services' review of all public input indicated 
the most acceptable alternative was a scenic route over the 
existing road system. This decision was due in large part to the 
findings of the study which showed increasing resistance to 
major construction changes which would expend huge sums of 

money and disrupt the area's eco-system. This fact along 
with an increasing need for economical help and increased usage 
of the Shawnee helped make the decision. 
Findings of this study supporting the decision included increasing 
positive attitudes toward the alternative related to concerns for 
water recreation, nature recreation, protection of the environment, 
community needs, individual needs, education, and the politics of 
the situation. 
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Summary and Conclusions 

We have defined content analysis as any systematic and objective 
technique which can reduce Into smaller sub-parts, existing communications. 
It Is the analytical reduction of a text to a standard set of 
statistically manlpultable symbols representing the presence. Intensity, 
or frequency of characteristics. 

The procedures used In this study show content analysis can 
be a reliable way to make currently existing written materials more 
useable. The code sheet was developed specifically for the purpose 
of helping the Forest Service make a decision on alternatives. The 
procedure did not correspond directly to content analysis procedures 
advocated by journalism and communications media disciplines, and 
It should not. Instead, by developing a code sheet Indicating the 
specific things to be analyzed by a panel of judges, the procedure 
can be adapted and changed to meet the various purposes we have In 
research and evaluation. 

What are these purposes? How useful Is the general procedure of 
content analysis to adult education research? We feel the usefulness 
of the procedures depend on knowing your specific purposes. We 
found four general purposes were reached In this case study. These 
four purposes are presented In Table 5 with our specific experiences, 
plus other examples. 
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In using content analysis, we can combine it with other empirical 
measurement techniques such as surveys, knowledge tests, attitude scales, 
and observations • In many cases, however, content analysis is a 
technique which can be a useful substitute for these other methods, 
especially when much existing but unorganized data already exists. 

In adult education we have at our disposal numerous sources of 
data for our research, which can be quantified by content analysis. 
These sources include logs and diaries, field reports, newspapers 
and magazines, transcripts of meetings and formal programs, management 
information, narrative reports, legal docuemtns (such as bills, 
resolutions, statutes, codes, and laws), petitions, inter- and 
intra-office communications and historical documents. 

In summary, sometimes the data we have provide information roughly 
in the form required. However, many other times, particularly when 
we seek to measure subjective aspects of clientele phenomena, such 
as the current values or opinions in a population, we must begin with 
text meaningless for our purposes that must somehow be processed 
to produce the classifications we require. At these times we see 
content analysis as a reliable and valid research method, central and 
not merely peripheral, to the development of adult education. This 
is especially true when we are sure of our purpose, have simple 
categories, explicit instructions, and well-trained analyzers. 
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